<html>
<head><meta charset="utf-8"><title>queries / futures · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html">queries / futures</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="147825603"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147825603" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147825603">(Nov 16 2018 at 15:59)</a>:</h4>
<p>Based on the talk of queries over in the steering meeting, I'm curious if:</p>
<p>1. Could the compiler's query system be written in terms of futures<br>
2. Would there be any benefit to it, if so?</p>



<a name="147825673"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147825673" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147825673">(Nov 16 2018 at 15:59)</a>:</h4>
<blockquote>
<p>1. Could the compiler's query system be written in terms of futures</p>
</blockquote>
<p>Dubious, almost no work compiler does is asynchronous.</p>



<a name="147825766"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147825766" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147825766">(Nov 16 2018 at 16:00)</a>:</h4>
<p>yeah, the compiler would have to have something useful to do while it's waiting for a future to finish.</p>



<a name="147826577"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826577" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826577">(Nov 16 2018 at 16:10)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> / <span class="user-mention" data-user-id="119009">@eddyb</span> "smaller" queries were mentioned during the meeting</p>



<a name="147826587"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826587" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826587">(Nov 16 2018 at 16:10)</a>:</h4>
<p>can you elaborate on what was meant by “smaller”?</p>



<a name="147826692"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826692" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826692">(Nov 16 2018 at 16:12)</a>:</h4>
<p>finer-grained, I assume?</p>



<a name="147826698"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826698" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826698">(Nov 16 2018 at 16:12)</a>:</h4>
<p>yes, esp. since we want to make at least some of them "larger"</p>



<a name="147826728"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826728" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826728">(Nov 16 2018 at 16:12)</a>:</h4>
<p><a href="https://internals.rust-lang.org/t/next-steps-for-reducing-overall-compilation-time/8429/4?u=michaelwoerister" target="_blank" title="https://internals.rust-lang.org/t/next-steps-for-reducing-overall-compilation-time/8429/4?u=michaelwoerister">https://internals.rust-lang.org/t/next-steps-for-reducing-overall-compilation-time/8429/4?u=michaelwoerister</a></p>



<a name="147826773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826773">(Nov 16 2018 at 16:13)</a>:</h4>
<p>So, for example I’m making a query currently to collect <code>#[optimize]</code> attributes from all the items in a crate, which essentially ends up touching all the items in a crate.</p>



<a name="147826776"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826776" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826776">(Nov 16 2018 at 16:13)</a>:</h4>
<p>how would one make it more fine-grained?</p>



<a name="147826889"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147826889" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147826889">(Nov 16 2018 at 16:14)</a>:</h4>
<p>I can’t see us being able to avoid such queries any way…</p>



<a name="147827650"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147827650" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147827650">(Nov 16 2018 at 16:24)</a>:</h4>
<p>what exactly is the purpose of this query?</p>



<a name="147828489"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147828489" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147828489">(Nov 16 2018 at 16:37)</a>:</h4>
<p>Don't we usually have a query to get such info for just one item and then have the collector run that query on all items yo cache the result if the result can only be computed in the current query</p>



<a name="147828678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147828678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147828678">(Nov 16 2018 at 16:40)</a>:</h4>
<p><span class="user-mention" data-user-id="124288">@Oli</span> yes, that makes sense if each individual query is expensive</p>



<a name="147840593"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147840593" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147840593">(Nov 16 2018 at 19:38)</a>:</h4>
<p><span class="user-mention" data-user-id="116155">@Jake Goulding</span> I think the main reason to do that would be as part of parallel execution — if you have two threads that both want to compute the type-check of X at the same time, you presently have to block (or else duplicate the work). I don't think it's worthwhile though.</p>



<a name="147840616"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147840616" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147840616">(Nov 16 2018 at 19:39)</a>:</h4>
<p>but once we have a working parallel compilation implementation this is the kind of thing we can explore</p>



<a name="147840628"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147840628" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147840628">(Nov 16 2018 at 19:39)</a>:</h4>
<p>(that is, trying to get some data on how much these sorts of conflicts arise in practice)</p>



<a name="147851663"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/queries%20/%20futures/near/147851663" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/queries.20.2F.20futures.html#147851663">(Nov 16 2018 at 22:46)</a>:</h4>
<p>parallel queries already provides asynchronous execution of queries -- I guess it wouldn't be too hard to provide a futures-like api in addition to the blocking one. But I also think that the rayon-like par_iter() api covers 95% of the use-cases pretty well.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>